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ABSTRACT 

With certifying end-of-pr ogram achievement as their 
goal, a group of educators conducted a pilot assessment program at 
Northeast Missouri State University in 1992. The program called for 
the videotaping of each student's third speech to be assessed by 
outside judges. The present study attempted to determine if 
videotaping and s elf -evaluat i on exercises earlier in the course would 
affect the outcome of the final assessment; the study, in other 
words, viewed assessment itself as an organic part of the learning 
process. Specifically, the study looked at three variables: would 
student performance increase if (1) he or she viewed an "excellent" 
student speech on video early in the semester? (2) if he or she 
viewed him- or herself on video earlier in the semester and was 
encouraged to critique him~ or herself? (3) and if he or she were 
evaluated throughout the semester on the same form that would be used 
for the end-of-the -semes ter assessment? Three educators, each 
teaching two sections of the basic course, participated in the study. 
The above variables were implemented in one course section of each of 
the educators and not in the other. Results showed no significant 
difference between the two groups in the outcome of the final 
assessment. While the results of this study cast serious doubts on 
the effectiveness of the three teaching strategies above, it suggests 
that the process of formulating assessment methods is also a process 
of researching the learning process. (Contains 17 references and 
three tables of data.) (TB) 
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Assessment Is Epistemic and Heuristic: 
The Role of Videotape in the Public Speaking Course 



Interest in educational assessment of higher education 
outcomes corresponds with and, in some instances, is a reaction 
to growing public demand for a higher degree of accountability. 
With this persepective, assessment efforts have focused primarily 
on measuring skills and competencies for the purpose of certify- 
ing end-of -program achievement. 

With certifying end-of -program achievement as our goal, 
during the fall of 1992 the authors conducted a pilot assessment 
program at Northeast Missouri State University, We reported on 
that pilot at SCA' s 1993 convention in Miami, Florida, The 
design of the assessment pilot called for videotaping the third 
of four speeches of each student enrolled in the basic public 
speaking course. The videotaped speech was then evaluated by 
outside judges using the SCA Competent Speaker Form (Morreale, et 
al.) However, the potential use of this data, produced by 
videotaping each student delivering a speech and having each 
speech evaluated by outside judges using a standardized form, 
began to attract our attention and is the focus of this paper. 

As educational professionals, our task has always been to 
explore and increase our understanding of the learning process . 
However, it often seems futile to develop and modify content and 
strategies used in the classroom with only intuition telling us 
whether those efforts have been successful. An overlooked 
characteristic of assessment is its heuristic and epistemic 
nature. Once an assessment program reliably measuring education- 
al outcomes is in place, the potential data inspires a multitude 
of meaningful questions specific to how students learn certain 
skills, what teaching strategies are effective and what charac- 
teristics impact learning. 

Viewed in this context, assessment is not just an extra test 
to purchase, administer or take but also is essential as stu- 
dents, faculty and institutions carry out their work. Assessment 
is an organic part of learning. It can be an important means of 
providing information about the teaching and learning processes 
and in turn can be a means of improving their effectiveness. 

In the fall of 1993 the authors undertook a quasi-experimen- 
tal study using data collecting procedures developed for the 
earlier pilot project. Since our assessment project relied on 
videotaping it is not surprising that our first research ques- 
tions had to do with the effects of our assessment 
methodologies on student outcomes as measured by the evaluations 
:>f videotaped speeches. Specifically, we wondered whether 
speaking performance would improve if a student first viewed a 
videotaped model of an "excellent" student speech. We wondered 
whether speaking performance would improve if a student viewed 
him/herself giving a speech on videotape prior to the assessed 
speech. Finally, we wondered whether speaking performance would 
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improve if a student were trained on an evaluation form, rated on 
the same form and rated others on that form for all speeches 
prior to and including the assessed speech. The literature 
review revealed a number of studies relating to these three 
questions . 

In a comprehensive survey of how public speaking teachers 
use model speeches in the classroom, Ronald Matlon delineates the 
following seven-pronged rationale : 1) to demonstrate the abili- 
ties of great speakers, 2) to increase the student ' s knowledge of 
the humanities, 3) to understand the strategic role speaking 
played in shaping history, 4) to teach students to be effective 
critics, 5) to increase interest in speech, 6) to encourage 
confidence and 7) to illustrate principles of public speaking (50- 
57) . Behavioral scientists have long 

realized the impact of observation of others on behavior. "Model- 
ing" studies suggest a plausible relationship between Matlon' s 
last two rationale, confidence level and understanding principles 
of public speaking, and a student's eventual speaking success. 

Carkof f suggests that "modeling, " observational learning, is 
the most significant source of learning (cited in Hosford 46) 
Bandura (Principles;" "Self -efficacy ; " "Social Learning") demon- 
strated that mere exposure to appropriate models promotes many 
kinds of human behavior, and behaviors already learned can be 
strengthened or extinguished by the same process* Through model- 
ing, a variety of skills and knowledge such as those present in 
quite difficult or complex behaviors often can be transmitted 
simultaneously instead of gradually shaping the desired behavior* 
On the other hand, studies (cited in Hosford) indicate that 
models perceived as prestigeful, attractive, high in status, 
powerful and ethnically similar promote greater imitative learn- 
ing than those presented as opposite or less accomplished in 
these dimensions (47) . Other studies suggest that personality 
characteristics may be significantly related to a person's 
predisposition to imitate or not imitate particular kinds of 
behavior (47) . Generally, the more perceptive and confident a 
person is, the more readily he/she will emulate both idealized 
models and those whose behavior is highly useful, (Bandura, 
"Social Learning") . A number of studies support the hypothesis 
that imitative learning is enhanced when the observer's perceived 
similarity to the model increases (cited in Hosford 47) . 

One form of observational learning which should promote 
identification with the mode] , both affectively and cognitively, 
is that of self -observation, i.e. having an individual observe 
instances of his/her own behavior. The literature is mixed 
relating to our second research question: do performance outcomes 
improve if a student views him/herself giving a speech on video- 
tape prior to the assessed speech? Mulac summarized that video- 
taped feedback caused a meaningful increase in student acquisi- 
tion of speech skill. "Students receiving videotape replay 
demonstrated significantly greater skill in oral communication at 
the end of the course than students receiving audiotape or no 
electronic replay" (214) . Miles concluded that using video 
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replay for student self -critique is of benefit primarily in 
assisting the student to identify and improve in language and 
delivery (283) . 

Bradley, on the other hand, found that speaking 
ability is not affected during a basic speech course by using the 
video-recorder to view speaking assignments in the classroom. 
Waggoner and Scheid (cited in Quigley and Nyquist) found that 
students did not improve their skills by simply viewing and 
rating videotapes of their past performances. 

However, according to Deihl, Breen and Larson "television 
playback seems to be a valuable aid to the instructor when he 
employs some type of criticism and comment with it" (188) . This 
qualified use of videotaped self -observation, accompanied with 
verbal evaluation, seems to be a point of agreement among re- 
searchers (Quigley and Nyquist 325) . 

From a psychological perspective, Hosford summarized studies 
showing self -observation experiences resulting in increased self- 
acceptance, jnore realistic self -appraisal , more openness to 
feedback from others and increased feelings of responsibility. 
Individuals who already feel confident about their behavior in a 
particular area experience less negative arousal from self- 
observation than do those having low self-esteem in th^.t area 
(52) . 

Finally, Dicker, Crane and Brown found students who repeat- 
edly observed themselves speaking increased their self ratings 
significantly less than comparable students using the same 
assignments without self -viewing . Self -viewing students did, 
however, show more correspondence to instructor ratings than did 
students without self -viewing (14 0) . 

Providing feedback as a form of speech evaluation for the 
purpose of improving future performance was an accepted teaching 
strategy long before the utility of videotape technology. 
However, the literature is sparse relating to our third research 
question: will performance outcomes improve if a student is 
trained on an evaluation form, rated on the same form and rated 
others on that form for all speeches prior to and including the 
assessed speech? Studies of the influence of written comments 
and/or ratings on subsequent performance show inconsistent 
results (cited in Booth-Butterf ield) . Some studies f?nd only weak 
and inconsistent effects with the use of written feedback. Others 
report increases in interest, motivation and achievement. Some 
studies noted that the effects of ccuments depend upon the 
initial performance level of the student (120) . Sorenson and 
Pickett found that rating ' others' behavior seems to be an effec- 
tive way to develop interview skills (18) . 

In discussing the effects of trait anxiety on written 
feedback, Booth-Butterf ield suggests that "if clear critiques are 
provided, anxiety does not appear to negatively influence percep- 
tions. It is the combination which tends to facilitate individu- 
als taking responsibility for their performances and undertaking 
appropriate change" (129) . 

We found no research relating to the impact of specific 



evaluation forms or formats on future performance. Indeed, it 
was a lack of standardized and psychometrically tested evaluation 
forms that prompted the SCA Committee for Assessment and Testing 
to develop The Competent Speaker Speech Form, a standardized and 
tested instrument for evaluating public speaking competency at 
the higher education level (Morreale, et al) . It is designed to 
serve a number of purposes including evaluating informative and 
persuasive speeches in class and as a tool for instructing and 
advising students (3). The form calls for lating eight distinct 
competencies and allows for written comments. 

Based ya this review we developed three hypotheses relating 
to three specific teaching strategies: 

1. Individuals who view a videotaped model of an "excellent" 
student speech will improve speaking performance more 
than those who do not view a model speech. 

2. Individuals who view a videotaped speech of self 
accompanied by instructor evaluation will 
improve speaking performance more than those who do 
not view a videotaped speech of self accompanied by 
instructor evaluation. 

3. Individuals using the same evaluation form, including 
ratings and written comments, for all speeches throughout 
a semester will improve speaking performance more than 
those who use multiple evaluation forms. 

Procedure 

Six sections (N = 122) of the basic speech course taught by 
three instructors (two sections per instructor) at Northeast 
Missouri State University were utilized for this experiment. 
Enrollment in these sections was predominantly freshr. n and 
sophomore students. All six sections used a public speaking text, 
and all students were required to give four speeches which were 
graded by the instructor. 

One section of each instructor was assigned to a treatment 
group and one to a comparison group. The treatment and comparison 
groups for each instructor were given the same specific speech 
assignments in the same sequence. All content was held constant 
for the two groups taught by the same instructor. Although 
classroom? and times differed between the groups for each in- 
structor, all were typical patterns to which students on the 
campus were accustomed > 

In treatment group 1, prior to having their own speech 
videotaped, students viewed a videotaped speech of student 
delivering a speech. This speech had been previously rated as 
excellent by four speech communication faculty on the SCA Compe- 
tent Speaker Form. Students were told that the speaker 
had won a university oratorical contest and were given an outline 
of the speech. Following the viewing, students and instructor 
critiqued the speech in an open class discussion. 

In treatment group 2, each student was videotaped while 
delivering an informative speech (second of four speeches) . 
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Within a week of his/her speech the student met privately with 
the instructor to view and critique his/her own videotaped 
speech. All conferences were concluded prior to the assessed 
videotaped speech. 

In treatment group 3, all students were introduced to the 
SCA Competent Speaker Form and trained in its use prior to their 
first speech. Throughout the semester each student was both a 
critic using the instrument to evaluate others and a receiver of 
evaluaiticn based on that form for all four speeches. This 
contrasted with the comparison group in which a different cri- 
tique form was used for each of the four speeches.. 

In all six sections the third of four speeches was video- 
taped. Three student workers were trained in the use of the 
camcorder and videotaped all students in the six sections. 
Camera angle, lens opening and camera distance were held constant 
for all speeches. Students in all sections were identified by a 
uniform numbering system on the videotape eliminating the possi- 
bility of student name and instructor recognition by judges. 

Later, each of the videotaped speeches was evaluated by 
three outside judges, two professionals with doctorates in 
communication and one lay person, over 18 years of age. A total 
of sixteen judges were trained on the SCA Competent Speaker Form. 
Training consisted of reviewing the standards and criteria for 
the eight competences. Review was based on materials provided by 
Morreale, et al . The videotape of exemplary speeches which 
accompanies the training material was not used. The final assess- 
ment rating for each student in all groups was determined by 
averaging scores of the three outside judges.. 

The dependent variable as indicated by the hypotheses was 
students' assessment rating of a speech as determined by SCA 
Competent Speaker Form. "Initial testing of The Competent 
Speaker Form indicated that the instrument is psychometrically 
sound in terms of reliability and validity. 11 (Morreale, et al . 
33) . 

The t test was selected as the appropriate statistic model 
to apply to the data comparing the mean of the treatment group 
with the mean of the concomitant comparison group for each 
hypothesis (Hays 327). The .01 level was required for signifi- 
cance of all tests of hypotheses. 

Results 

Table 1 presents the results of the t test of differences 
between treatment and comparison groups mean assessment scores 
for hypothesis one. 



Table 1 

Comparison of Assessment Scores 
Treatment 1: - Model Speech 





N 




SD 


t 


Treatment Group: 


20 


44 . 99 


5. 57 




Model Speech 


















2.36* 


Comparison Group: 


19 


61.67 


3 . 99 




No Model Speech 











* p=.02 



The mean assessment score for students viewing the model speech 
was 44.99: the comparison group was 61.67. The difference 
between the two means was not significant, thus hypothesis one 
was not supported. 

Table 2 presents the results of the t test of differences 
between treatment and comparison groups mean assessment scores 
for hypothesis two. 

Table 2 

Comparison of Assessment Scores 
Treatment 2 : - Self -Observation 





N 


X" 


SD 


t 


Treatment Group : 


22 


44 . 90 


4 .53 




Self -Observation 


















. 0014* 


Comparison Group: 


22 


44 .89 


4 .37 




No Self -Observation 











* p=.99 



The mean assessment score for students who viewed themselves 
on videotape tfas 44.90; the comparison group was 44.89. The 
difference between the two means was not significant, thus 
hypothesis two was not supported. 

Table 3 presents the results of the t test of differences 
between treatment and comparison groups mean assessment scores 
for hypothesis three. 
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Table 3 

Comparison of Assessment Scores 
Treatment 3: - Same Evaluation Form 



N X ~SD t 

Treatment Group: 18 47.31 4.34 

Same Evaluation Form 

.331* 

Comparison Group: 21 47.77 4.03 

Multiple Evaluation Forms 



p=.74 



The mean assessment score for students using the same evaluation 
form for all speeches was 47.31; the comparison group was 47.77. 
The difference between the two means was not significant, thus 
hypothesis three was not supported. 

Discussion 

The purpose of this study was to determine whether three 
teaching strategies: using an "excellent" model, self -observa- 
tion and using the same evaluation form, affect students speech 
performance as measured by SCA Competent Speaker Form. 

Individuals who viewed a videotaped model of an "excellent" 
student speech did not do better on subsequent speech performance 
than those who did not view a model speech. However, it should 
be noted that the comparison group's mean assessment score was 
actually higher than the treatment group at a .02 level of 
significance. This indicates that there were differences between 
the two groups that can not be attributed to chance nor to the 
treatment. This effect may be due to a lack of homogeneity or 
the samples. The effect of viewing a "model" speech on speakir.g 
performance should be studied further . 

Individuals who viewed a videotaped speech of self, accompa- 
nied by instructor evaluation, did not do better on subsequent 
speaking performances than those who did not experience self- 
observation. This result tends to support prior studies which 
suggest that the benefits of videotaped self -observation lie in 
the affective domain (Goldhaber and Kline; McCroskey and Lashbro- 
ok) . On the other hand, very few researchers have focused on the 
effects of self -observation on actual speaking performance. As 
more standardized measures of public speaking performance are 
developed this line of inquiry should continue. 

Individuals who used the same evaluation form throughout the 
semester did no better on subsequent public speaking performance 
than did those who used multiple evaluation forms. 

In addition to the implications for future research already 
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mentioned, an analysis of each of the eight competencies rated on 
the SCA Competent Speaker Form might reveal significant differ- 
ences for these treatments at more specific levels of speaking 
performance. For example, Miles found videotaped play back was 
useful for improving language and delivery only (382) . 

While the results of this study cast serious doubts on *" 1 2 
efficacy of these three strategies for improving public speaK^ng 
performance, the results provide important evidence that as we go 
about the business of developing instruments which accurately 
assess educational outcomes, including speaking performance, we 
are also developing a means for researching the learning process. 
Indeed, assessment is epistemic and heuristic. 
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